Learning to cooperate in multi-agent systems by combining Q-learning and evolutionary strategy

Cooperative games can represent interactions between multiple agents in many real-life situations. Thus single-stage cooperative games provide a stylized, abstracted environment for testing algorithms that allow artificial agents to learn to cooperate in such settings. Individual reinforcement learners often fail to learn coordinated behavior. Using an evolutionary approach to strategy selection can produce optimal joint behavior but may require significant computational effort. Our goal in this paper is to improve convergence to optimal behavior with reduced computational effort by combining learning and evolutionary techniques. In particular, we show that by letting agents learn in between generations in an evolutionary algorithm allows them to more consistently learn effective cooperative behavior even in difficult, stochastic environments. Our combined mechanism is a novel improvisation involving selecting actual rather than inherited behaviors.